HOW CAN WE MAXIMISE THE IMPACT OF OUR RESEARCH? 


A key to maximising the impact of research is to keep this goal in mind throughout the research process; to view it as an ongoing 
activity, not something tacked on at the end. The following tips are offered as ways that researchers can help to maximise the 
impact of research and the measurement of this impact. 

Dissemination 

Dissemination is crucial to ensuring that research has maximum impact; lack of attention to this stage will almost certainly 
compromise the effectiveness of your research. NCVER has found that personal contact with stakeholders and succinct products 
are particularly valuable. These short pieces, which are nevertheless challenging to produce, can be capstone pieces that bring 
together the key messages arising out of a body of work or a research program. 

Contacts: networks and discussions 

The human aspect of research is an important component in how it impacts on others. Face-to-face contact, networking and 
engaging with stakeholders have been shown as ways of increasing the effect of research. 

Opportunities to share information and interpret the results of the research to interested parties should always be considered as 
part of the research process. These meetings do not always have to be formal. 

Engaging end-users during the research process can help to provide a clear and effective direction for the research. They can 
be used to help define the research problem and to provide feedback at work-in-progress seminars and during the final stages 
of the project. This feedback can also be used as an indicator of initial impact. When engaging end-users, care must be taken to 
maintain the integrity and independence of the research. 

Knowledge translation 

Knowledge translation takes the research beyond dissemination activities. It actively seeks to interpret the research according 
to the needs of different audiences. Information from the research is distilled into a variety of different formats that can 
include good practice guides, research overviews for specific audiences, briefing papers, presentations and discussion forums, 
just to name a few. 

Building the knowledge translation stage into the process at the beginning of the research will maximise its effectiveness. 
Intermediaries who are skilled at knowledge translation can also assist in getting the message out to a wider audience. This does 
not necessarily have to be the people who conducted the research, although in many cases it will be. 

Tools to aid research impact 

Keeping records of impact during the research process, be they requests for information, presentations, or media interviews is 
a good discipline. These records are useful for assessing impact in terms of providing information on potential end-users. The 
information can also be a guide to dissemination as it focuses on who are the interested stakeholders. 

The recording tool should be simple to use. For example, a spreadsheet or database keeping track of types of contact such as 
requests for information, requests to present, the contact organisation/person, date, who was contacted, and how useful the 
contact was should suffice. 
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MEASURING AND MAXIMISING 
RESEARCH IMPACT IN APPLIED SOCIAL 
SCIENCE RESEARCH SETTINGS 




This guide describes the National Centre for Vocational Education Research (NCVER) approach to measuring impact using 
examples from our own case studies, as well as showing how to maximise the impact of applied social science research. 

Applied social science research needs to demonstrate that it is relevant and useful both to public policy and practitioners. In 
this current economic climate, researchers and their organisations need to be able to justify public expenditure on research. As 
such, it is important to evaluate the impact of research in line with its aims, be they policy- or practice-oriented. 

For these reasons, NCVER has embarked on measuring the impact of its research. The findings and lessons learned from previous 
impact assessments have provided some pointers on both how to maximise and measure the impact of applied research. While 
our examples are specific to the field of education and training, they can be applied to other social science settings. Our 
methodology has been influenced by work from the health sector, with NCVER adopting the ‘payback framework’ developed by 
Buxton and Hanney (1996). 

Impact can be defined as the application, use and influence of research across various domains. In our context, the four domains 
considered are producing knowledge, building the capacity of researchers and end-users of the research, informing policy, and 
informing practice. 

HOW DO WE ASSESS IMPACT? 

While our aim is to maximise the impact of research, it is also important for researchers or their organisations to demonstrate its 
impact. To do this, we need to be able to measure this impact in some way. 

Measuring impact is not a straightforward process and the process is limited, particularly when trying to attribute causality. 

For instance, how does one judge the impact of a particular piece of research on policy, given the complex nature of the policy 
process and the range of competing information and other factors which influence it? There are also implications for the timing 
of measuring impact, in that few impacts are realised in the short term. Our studies suggest an optimal time to conduct an 
impact assessment is two to three years after publication of the initial research report. 

Approaches to measuring impact need to be pragmatic and realistic. The primary aim or purpose of the research as well as the 
concerns of potential end-users set the foundation for our impact studies; and the four domains act as a contextual model. 

An effective way to measure impact in the social sciences is to use a combination of bibliometric data and case study material, 
which includes interviews with other researchers and people who have used the research. Bibliometric data primarily provides 
information on citations, which is an indicator of engagement with the research. By talking to end-users and researchers 
themselves, it is possible to gather information about how the research has been used and what influence it has had. 

It’s therefore not just about the ‘numbers’ when recording and recognising impact in the applied social sciences; it is also about 
the willingness of the researcher to engage purposefully with their findings. In this sense, a richer context around research 
impact is developed rather than relying on the number of citations that a piece of work receives. 

Judgment is required in how many resources are devoted to uncovering the impact of a piece of research as this can be expensive 
and time-consuming, especially when end-users prove difficult to locate. In addition, when they are found, end-users may be less 
familiar with the research than first anticipated and need considerable briefing to uncover where it has been used less explicitly. 

This good practice guide is based on the following reports: 

• Assessing the impact of research: a case study of the LSAY Research and Innovation Expansion Fund, by J Hargreaves 

• Assessing the impact of NCVER’s research, by J Stanwick, J Hargreaves, & F Beddie. 
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WHAT KIND OF IMPACTS CAN BE FOUND? 


Lessons for impact 


There are many ways in 
which a variety of outputs 
from a piece of research 
can have a wide-ranging 
impact. In applied 
social science research, 
citations in journals 
may not be as important 
as media citations and 
citations in legislation. 

No one citation tool is 
comprehensive, so a 
variety of search tools 
should be used. 


Producing knowledge 

Using a more ‘traditional’ notion of impact which includes not only outputs of 
the research such as formal reports, but also citations in other research, the 
media, parliamentary databases or legislation, and web downloads 

Knowledge production can be divided into two categories: outputs of the 
research and citations. 

NCVER’s impact studies identified a variety of outputs including research reports, 
conference papers, discussion forums, podcasts and syntheses. 

Citations occur through a variety of mechanisms. These include journal articles, 
reports, book chapters, conference papers, media and parliamentary databases 
or legislation. Media is a good indicator of whether a topic resonates with a 
broader audience, but is not necessarily a clear-cut impact measure. Citations 
in legislation are important because they provide a direct indication of how 
research has entered policy development. There are various tools available for 
citation analysis. These include Google, Google Scholar, Publish or Perish 
<www.harzing.com/pop.htm>, Scopus <www.scopus.com/home.url> and 
parliamentary databases. 


Building capacity 

Assessing the extent to which the output has led to further research, 
developed early career researchers and improved the capacity of 
researchers and stakeholders to engage with the material 

Capacity building can be examined from two perspectives: that of the 
researcher and that of the end-user. For the researcher, this may mean 
professional development, opportunities to undertake further research, and 
advancement in their career. For the end-user, this may mean greater awareness 
of, and ability to translate, research findings. 

An impact assessment (Hargreaves 2012) of the Longitudinal Surveys of 
Australian Youth (LSAY) Research and Innovation Expansion Fund demonstrates 
a variety of capacity-building impacts that can be achieved from research. The 
aim of this particular fund was to broaden the use of LSAY data for research 
into youth transitions, as well as expand its reach within the public policy 
arena. Examples of measures of capacity-building impacts from a researcher 
perspective were undertaking successful applications for further research, and generating more targeted research 
for the future. From an end-users’ perspective, capacity-building impacts included demonstrating the value of LSAY 
to others, and having a greater awareness of how the data could be used as a result of data workshops and a related 
youth policy forum. 

In the earlier impact assessment by Stanwick, Hargreaves and Beddie (2009), capacity-building impacts were found 
for a research report which examined strategies undertaken by adults who, despite having significant difficulties 
with literacy, were successful in life and work. This work led to the researchers developing skills and capabilities in 
multi-media which enabled them to win tenders for further work requiring those specialist skills. 


Lessons for impact 


Capacity-building impacts 
such as the research 
generating further work 
are often incidental; 
however, impacts such 
as holding forums or 
workshops which will 
increase end-users’ 
capacity to access and 
use information can also 
be built into the research 
process. 
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Lessons for impact 


The impact on policy is 
unlikely to be direct, 
but it is still important 
to recognise the role 
that applied research 
plays in informing policy. 
Encouraging researchers 
or knowledge brokers 
to engage directly with 
policy-makers helps in the 
distillation of research 
findings. Governments 
need to consider 
transparent referencing 
of research sources for 
their policies. Where 
feasible, consideration 
should be given to planning 
research around themes 
noting, however, that a 
single report on a particular 
topic is sometimes the most 
appropriate approach. 


Informing policy 

Assessing whether the research has been useful in informing or guiding policy 

Stanwick, Hargreaves and Beddie (2009) caution that the process of influencing 
policy is not straightforward. There are few direct links with the research, which 
often becomes conflated with other sources of information. This is also compounded 
by lack of good referencing in policy documents. Sometimes the process of influence 
almost seems to occur by osmosis. Research can provide new information, or 
validate or raise questions about existing policies. It can also give people the 
underpinning knowledge to debate and engage with policy discussions and decisions. 

Policy impacts are often not visible in the public domain. In one example, 
research on apprenticeship completion rates was used by a federal government 
department to guide internal discussion and policy directions. This impact was 
uncovered by talking to government representatives and would not have been 
identified in publications available in the public domain. 

It is worth noting that generally a body of research has more impact on policy than 
a single piece of work. Considering a topic or question from different perspectives 
or in a range of formats offers a variety of evidence, perhaps with more weight, 
from which policy-makers can draw conclusions. Sometimes the body of work 
contains a synthesis piece of the key findings, as was the case in research relating 
to recognition of prior learning (Stanwick, Hargreaves 6t Beddie 2009). 

Examples of policy impacts from our case studies include the Research and 
Innovation Expansion Fund informing policies on rural and remote education as 
well as contributing to a policy framework on youth transitions (Hargreaves 2012) 
and an Indigenous systematic review used by a federal government department to 
guide policies in Indigenous engagement (Stanwick, Hargreaves & Beddie 2009). 


Lessons for impact 


Consideration of who the 
practitioner end-users 
are at the beginning of 
the research, where 
relevant, will make it 
more likely to influence 
practice. The accessibility 
of the research findings 
can facilitate impacts 
on practice. This is 
coupled with the ability of 
researchers or knowledge 
brokers to share research 
findings in practical and 
engaging ways. 


Informing practice 

Assessing whether the research has been useful in informing or guiding 
practice 

As with policy, the impact on practice can be tricky to detect. Aside from the 
sheer number of practitioners to survey, it can be difficult to identify direct 
causality. However, careful consideration of who the practitioner end-users are 
can uncover some of these impacts. Identifying possible end-users is a process 
which can occur at the beginning of the research. It should also be recognised 
that some research is not designed to have an impact on practice. 

Stanwick, Hargreaves and Beddie (2009) found that a systematic review of 
what training works for Indigenous people had an impact on practice in that it 
was used to develop the selection criteria for subsequent training programs for 
Indigenous people. 

Hargreaves (2012) found that the Research and Innovation Expansion Fund 
research informed the practical steps taken to support students at a trade 
training centre. In particular, the research influenced the way partnerships 
between training providers and regional schools could be established and 
operated. 
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